Part 1: Fun with Filters

1.1: Finite Difference Operator

We will start with the initial derivative based edge finding solution for cameraman, using $dx = [[1, -1], [0,0]], dy=[[1, 0], [-1,0]]$

<matplotlib.image.AxesImage at 0x13a948490>
<matplotlib.image.AxesImage at 0x1353dc410>
<matplotlib.image.AxesImage at 0x13adda050>

We can extract the edges after thresholding out some of the noise.

<matplotlib.image.AxesImage at 0x13aa3e9d0>

1.2 Derivative of Gaussian (DoG) Filter

Now we will blur the image and see how our previous process compares.

<matplotlib.image.AxesImage at 0x13aa94f50>
<matplotlib.image.AxesImage at 0x13b216e10>

With the blurred image, our regular dx/dy convolutions highlight the edges with almost all of the noise removed, resulting in more defined shapes.

We can also do this via taking the derivatives of the Gaussian and applying those, to get the same result. Let's first see the derivates of the Gaussian.

<matplotlib.image.AxesImage at 0x13b27e790>
<matplotlib.image.AxesImage at 0x13b335ed0>

Now applying them to the image individually.

<matplotlib.image.AxesImage at 0x13b3f3a10>
<matplotlib.image.AxesImage at 0x13b49fc50>

Now applying them together finally we see the edges.

<matplotlib.image.AxesImage at 0x13b5acc50>

The two results are the same.

1.3: Image Straightening

We will start by analyzing the sample image

(2849, 3799)

To understand how this should be done, lets first take a look at the individual derivatives and the gradient angles of the original image. We can calculate the gradient angles as simply the arctan(dx, dy).

<matplotlib.image.AxesImage at 0x135437ed0>
<matplotlib.image.AxesImage at 0x1354ed610>
/usr/local/lib/python3.7/site-packages/skimage/io/_plugins/matplotlib_plugin.py:150: UserWarning: Float image out of standard range; displaying image with stretched contrast.
  lo, hi, cmap = _get_display_range(image)
<matplotlib.image.AxesImage at 0x1355b1350>

The gradient angles are too noisy to meaningfully understand, so let's try filtering by a threshold on the magnitude of the angles, which conceptually tells us the strength of that angle as a signal to filter on. We can calculate the magnitudes as a function $\sqrt{dx^{2}+{dy}^2}$

/usr/local/lib/python3.7/site-packages/skimage/io/_plugins/matplotlib_plugin.py:150: UserWarning: Low image data range; displaying image with stretched contrast.
  lo, hi, cmap = _get_display_range(image)
<matplotlib.image.AxesImage at 0x1356678d0>

Now we can look at the filtered angles.

<matplotlib.image.AxesImage at 0x13571bb90>

Histogram of original

As requested, we now compute the histrogram of the original image's gradient angles

Now let's try to understand what the verticle and horizontal lines look like in this image

There are 141448 verticle edges
There are 111407 horizontal edges
There are 3525614 edges total

Now lets define the number of edges to be the count of these values taken as a ratio between the count of the total number of total edges

We can now find the best angle by iterating over a large range of angles and picking the one with the greatest proporation of straight edges

We find the best angle is -3, so we can now show what that is supposed to be.

<matplotlib.image.AxesImage at 0x13ab65b90>

As well as the histogram for this

Now to do this on 3 other images:

<matplotlib.image.AxesImage at 0x13ae66110>
Straightened by 10 degrees
<matplotlib.image.AxesImage at 0x13aebc6d0>
<matplotlib.image.AxesImage at 0x13af7a310>
Straightened by -5 degrees
<matplotlib.image.AxesImage at 0x13afe31d0>
<matplotlib.image.AxesImage at 0x13b087910>
Straightened by 7 degrees
<matplotlib.image.AxesImage at 0x13ad8d8d0>

This last one is clearly a failure since the actual tower is tilted. Interesting, we can see that the tower is tilted by about 7 degrees.

Part 2: Fun with Frequencies!

2.1: Image "Sharpening"

Original Taj Photo

(300, 300, 3)
<matplotlib.image.AxesImage at 0x131ea5250>

Sharpened Taj Photo

<matplotlib.image.AxesImage at 0x131f10350>

Sharpening Obama

Original Obama
Sharpened Obama

Blur then Sharpen (cam.png)

Original Random photo

(604, 708, 3)
<matplotlib.image.AxesImage at 0x131f76290>

Blurred Random photo

<matplotlib.image.AxesImage at 0x131fdea90>

Resharpened Random photo

<matplotlib.image.AxesImage at 0x13204d790>

The resharpened image is quite a bit closer to the original, but not quite there. You can see the eyes in more detail, but much was lost in the skin.

2.2: Hybrid Images

Now we're testing hybrid images, first with the sample images.

<matplotlib.image.AxesImage at 0x1320bb7d0>
<matplotlib.image.AxesImage at 0x13211f990>
<matplotlib.image.AxesImage at 0x1371fcbd0>
<matplotlib.image.AxesImage at 0x136ca7a90>
<matplotlib.image.AxesImage at 0x1374a3710>

Next up, my friends Cam and Sara

(944, 820, 3) (980, 794, 3)
<matplotlib.image.AxesImage at 0x13750e590>
<matplotlib.image.AxesImage at 0x13756e310>
Using matplotlib backend: TkAgg
Please select 2 points in each image for alignment.
Shape 0: (944, 820, 3), (980, 794, 3)
Shape 1: (1455, 923, 3), (1399, 809, 3)
Shape 2: (1249, 792, 3), (1399, 809, 3)
Shape 3: (1249, 792, 3), (1399, 809, 3)
<matplotlib.image.AxesImage at 0x137b2e490>

I would consider this a failure, since you can't really tell the two apart at any distance, and the teeth as a feature just make the image hard to visually process.

Next is Obama

Using matplotlib backend: TkAgg
Please select 2 points in each image for alignment.
Shape 0: (309, 257, 3), (822, 617, 3)
Shape 1: (329, 283, 3), (871, 639, 3)
Shape 2: (329, 283, 3), (360, 264, 3)
Shape 3: (329, 283, 3), (360, 264, 3)
<matplotlib.image.AxesImage at 0x138ad61d0>
<matplotlib.image.AxesImage at 0x138645290>
<matplotlib.image.AxesImage at 0x13b968310>

This one is my favorite, so I'll break down the fourier analysis for this.

Original Fourier Transform Magnitudes:

Obama old Fourier
<matplotlib.image.AxesImage at 0x13a50b390>
Obama young Fourier
<matplotlib.image.AxesImage at 0x13a4a5550>

Lowpass FT Mag:

<matplotlib.image.AxesImage at 0x134f71c10>
<matplotlib.image.AxesImage at 0x135039ad0>

Highpass FT Mag:

<matplotlib.image.AxesImage at 0x1353210d0>
<matplotlib.image.AxesImage at 0x1352bead0>

This Fourier graph is a little weird, considering we should see a hole where the low frequencies are... That being said we can look at the actual highpass-filtred image and see that it is functioning correctly so not sure what's going on here.

2.3: Gaussian and Laplacian Stacks

Starting with the lincoln image

<matplotlib.image.AxesImage at 0x13b6222d0>

Trying it out on the Obama image

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

2.4: Multiresolution Blending

First we test out the oraple:

<matplotlib.image.AxesImage at 0x13ce11690>
<matplotlib.image.AxesImage at 0x13ce76190>
<matplotlib.image.AxesImage at 0x13ced5b10>
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
<matplotlib.image.AxesImage at 0x13cf3c690>

And now, we'll see two faces from the recent news in a new light...

(300, 300, 3)
<matplotlib.image.AxesImage at 0x13ddac8d0>
<matplotlib.image.AxesImage at 0x13de15390>
<matplotlib.image.AxesImage at 0x13de67dd0>

This one was my favorite

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
<matplotlib.image.AxesImage at 0x13e54fa10>
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
<matplotlib.image.AxesImage at 0x13e6029d0>

And now for a cute image

<matplotlib.image.AxesImage at 0x13e6c4210>
<matplotlib.image.AxesImage at 0x13e7294d0>
<matplotlib.image.AxesImage at 0x13eb5be90>
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
<matplotlib.image.AxesImage at 0x13e8abd10>